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Stochastic  Filtering  and  Control  of 
Linear  Systems:  A General  Theory 


A.  V.  Balakrishnan 

University  of  California  Los  Angeles 


A large  class  of  filtering  and  control  problems  for  linear  systems 
can  be  described  as  follows.  We  have  an  observed  (stochastic)  process 
y (t)  (say,  an  m x 1 vector),  t representing  continuous  time, 

0 < t < T < 00  . This  process  has  the  structure: 

y ( t)  = v(t)  + nQ(t) 

where  n^Ct)  is  the  unavoidable  measurement  error  modelled  as  a white 
Gaussian  noise  process  of  known  spectral  density  matrix,  taken  as  tire 
Identity  matrix  for  simplicity  of  notation.  The  output  v(t)  is  composed 

of  two  parts:  the  response  to  the  control  input  u(t)  and  a random 

'disturbance'  nT (t)  (sometimes  referred  to  as  'load  disturbance'  or 
'stale  noise')  also  modelled  as  stationary  Gaussian;  we  also  assume  the 
system  responding  to  the  control  is  linear  and  time-invariant  so  that  we 
have : 

5-t 

v(t)  = | B(t-s)  u(s)ds  + n.  (t) 

Jo 

where  uCO  is  always  assumed  to  be  locally  square  integrable,  and 

~f~  ^e:c<vKc(>  Sv^pfltrt~ecl.  /*n  p{\ KrcJe*'  no-  7 3 - <2- d- 9 2. 

M c\it-  e^a  t”i  c / .On/ ii i'w.-  , ft  F'-OS  £ ) ijp p. 


where  B(*)  is  a ’rectangular'  matrix  function  and 


dt  < oo  . 


We  assure  further  more  that  the  random  disturbance  is  ' physically  realizable' 
so  that  we  can  exploit  the  representation: 


nL(t>  - r 


F(t-p)  N(p)  dp 


where  F(P)  is  a rectangular  matrix  such  that 


[ 1 1 F(s)  [ | 2ds  <°° 

Jn 


where,  in  the  usual  notation, 


| | A | | = Tr.  AA* . 


We  assume  that  the  process  n^ (t)  is  independent  of  the  observation 
noise  process  nn(t). 


It  is  more  convenient  now  to  rewrite  the  total  representation  as: 


y(t,w)  = v(t,a>)  + Gw(t) 


:,w  . r 

Jo 


B(t-s)  u(s)ds  + I ^(t-s)  io(s)ds 

Jo 


where 


GG*  = I 
^"(t)G*  = 0 


o(  •)  is  white  noise  process  in  the  appropriate  product  Euclidean  space, 


I 1 1 & (t) I |2dt  <» 


' 


in  the  white  noise  formalism. 


This  problem  embraces  already  all  the  stochastic  control  problems 
for  systems  governed  by  ordinary  differential  equations  by  taking  the 
special  case  where  the  Laplace  transforms  of  B(  • ) and  &"(.-) 
are  rational.  But  it  also  includes  a wide  variety  of  problems  involving 
partial  differential  equations  where  the  observation  process  Y(t) 
for  each  t has  its  range  in  a finite  dimensional  Euclidean  space 
(measurements  at  a finite  number  of  points  in  the  domain  or  on  the  boundary 
for  example).  One  may  argue  that  any  physical  measurement  must  be  finite 
dimensional;  in  any  case,  the  extension  to  the  infinite  dimensional  case 
brings  little  that  is  new,  and  we  shall  not  go  into  it  here. 

As  a simple  example  of  a non-rational  case  we  may  mention: 


F(t)  = t"?/2  e Vt 

arising  fron  boundary  input  in  a half-infinite  rod  [5].  Note  that  the 
associated  process  n^(t)  is  not  'Markovian'  even  in  the  extended 
sense  [2]. 


(1.4) 


To  solve  our  problem,  our  basic  technique  is  to  create  an  'artificial* 
state  space  representation  for  (1.1).  It  is  artificial  in  the  sense  that 
it  has  nothing  to  do  with  the  actual  state  space  that  originates  with  the 
problems.  We  shall  illustrate  this  with  a specific  example  be  law.  Without 
going  into  the  system  theoretic  aspects  involved,  let  us  simply  note  that 
the  controllable  part  of  the  original  state  space  can  be  put  in  one-to-one 
correspondence  with  the  controllable  part  of  the  artificial  state  space. 


Let  H denote  L^lO ,°°  ;I^]  where  m is  the  dimension  of 

the  observation  process.  Let  A denote  the  operator  with  domain  in  H: 

0(A)  = l fell  | /( • ) is  absolutely  continuous  with  derivative 
f^(*)  e H also], 


Af  = r 


Let  B denote  the  operator  mapping  the  Euclidean  space  in  which  the 
controls  range5 into  H by: 


B u(t)  ~ B(£)u(t)  , 0 < C < oo 


and  similarly 


^ui(t)  ~ «>(C)ai(t)  0 < r,  < oo 


Assume  now  that  F(t)  and  B(t)  are  'locally'  continuous,  in 
0 <_  t < «°.  Then  we  claim  that  (1.1)  is  representable  as  C°- 

x(t)  = A x(t)  + Bu(t)  + J^wCt)  ; x(0)  =0.  ) 

y(t)  = C x(t)  + Gco(t)  ) 


(or  appropriate  'Wiener-process'  version),  where  C is  the  operator 


defined  by: 


Domain  of  C = LfeH  I f(t)  is  continuous  in  0 < t < »] 


(1.5) 


[or,  f(*)  is  'locally'  continuous]  and 


Cf  = /( 0) 


[value  at  the  origin  of  the  'continuous  function'  representative 

of  f(*)]. 

We  can  readily  show  that  x(t)  is  in  the  domain  of  C because  of  the 

assumption^ot  local  continuity.  On  the  other  hand  we  do  not  need  to  make 
the  'exponential  rate  of  growth ' assumptions  as  in  the  earlier  version  of 
the  representation  [ 3 ].  To  see  this  we  have  only  to  note  that  (1.5)  has 
the  solution,  (assuming  that  u( • ) is  locally  square  integrable) : 


=i: 


i: 


x(t)  = S(t-a)Bu(a)da  + | S(t-o)  ^'ai(a)da 

’o 

where  S(t)  is  the  semigroup  generated  by  A . Now 


(1.5) 


h(t)  = S(t-a)  Bu(cr)da  is  the  function: 


■c 

■ i: 


h(t,£)  = B(c+t-cr)  u(a)da  0 < t;  < » 

'o 


and  h(t,c)  is  locally  continuous  in  0 <_  £ < 00 , because  of  the 

local  continuity  of  B(*).  Hence  h(t)  is  in  the  domain  of  C,  for  each. 

Moreover 


C h(t) 


■ c 


B (t-a)  u(o)da 


Similarly 


-6- 


I 


I 


C S(t-a)  :>oj(o)do 

J0 


.^(t-a)aj(a)da 


which  suffices  to  prove  the  representation.  Of  course  to  complete  the 
representation  we  have  that  the  cost  functional  (1.2)  can  be  written: 


E [ [QCx(t),  QCx(t)]dt  + E [ [u(t) , u(t)]dt 

Jn  J n 


Cl. 6) 


In  this  fcm  we  have  a stochastic  control  problem  in  a Hilbert 
Space , and  we  may  apply  the  techniques  of  [ 4 ] ; except  for  the  complication 
that  C is  now  unbounded,  uncloseable.  The  'operators'  B and  8F  are 
Hilbert-Schmidt  and  in  this  sense  there  is  a simplification. 


Even  though  C is  uncloseable,  let  us  note  that 


Cx(t) 


■£ 


B(t-o)  u(a)da  + ^F(t-a)u)(a)da 


and  hence  is  actually  locally  continuous  in  0 < t,  and 


rP 

g(p)  = [ C S(p-o)  ,>/(a)da  0 < p < t 

J0 


defines  a linear  bounded  trans formation  on 


Wn(t)  = I,(<0,t).Rn) 

where  Rn  is  the  Euclidean  space  in  which  w(t)  ranges,  into 

wo(t)  = L2((0’t);RJ 


for  each  0 < t.  V'e  shall  only  consider  u(t)  such  that 
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tit 


- 


= f L (t,s 

Jn 


u(t)  = L (t,s)  y(s)ds  0 < t < T 
'0 


wliere 


■£ 


g(t)  = L (t,s)  f(s)ds  0 < t < T 


defines  a Hilbert-Schmidt  operator  mapping  W^(T)  into 
where 


WC(T)  = L2C(0,T);  R ] 


where  R is  the  real  Euclidean  space  in  which  u(t) 

P 

every  t . The  Hilbert-Schiridtness  implies  that  L(t,s) 

Schmidt  also  a.  e.  and  that 


T t 


f f ||L(t,s)||2  dt 
JQ  J 0 H-S 


< CO 


It  is  not  difficult  to  see  that 


u(t)  = I L(t,s)y(s)ds 
'0 


x(t) 


= f L(t, 
J n 

■c 


£ 


S(t-a)  B u(a)da  + S(t-a)  ^co(a)da 


y(t)  = C x(t)  + G aiCt) 


defiries  >:(•  ) uniquely,  for  each  u(  •) . 


ucm 


ranges  for 
is  Hilbert- 


r 


2.  The  Filtering  Problem. 

Let  us  first  consider  the  filtering  problem  for  (1.1)  taking 
u(*)  to  be  identically  zero.  We  shall  see  that  this  is  an  essential  step 
in  solving  the  control  problem.  Thus  let,  in  the  notation  of  Section  1, 


x(t 


->  ■ f 

> n 


S(t-c)  ,>(jj(a)dc 


(2.1) 


y4t,w)  = Cx(t,w)  + Gu>(t) 

As  we  have  noted  earlier,  the  only  difference  from  the  standard  problem 
treated  in  [4]  is  that  C is  uncloseable.  Nevertheless  since 


Cx(t 


,(d)  = [ 2F 
J n 


(t-a)  (jj(c)da 


and  is  continuous  in  t for  each  oi,  we  note  that,  denoting  by  y^(oj) 


the  element  in  W^(t)  defined  by 


y(s,w),  0 < s < t 

I 

we  see  that  y^Gu)  is  a weak  Gaussian  randan  variable  with  finite  second 
moment  in  W^Ct)  for  each  t.  Moreover  y^_  has  the  covariance 
operator: 

I + L(t)  L(t)* 

t 

where  L(t)  is  defined  by 


and  is  linear  bounded  on  VMt)  into  W (t);  and  I is  the  identity 
operator  on  W^(t) . Let 

x(t,w)  = E Cx(t,w)  | y^Cta)] 

Then  x(t,w)  belongs  to  the  domain  of  C for  each  t and  each  o> 
and  further 


C x(t,a))  = -E  [Cx(t,oo)  | y^Cu)]  (2.2) 

the  novelty  in  this  relation  arising  from  the  fact  that  C is  unbounded. 

'This  car:  be  seen  readily  as  follows.  We  note  that  (see  [4]) 


x(t,o))  = 


£ 

E [x(t,u>)  yt(u)  ] 


[I  + L(t)  L(t)  ] 


-1 


yt(w) 


(2.3) 


where 


* rt 

E [x(t,w)  yt(u)  ] / = j K(t,s)f (s)ds 


where 


f P f: 

K(t,p)  = S(t-p)  S(p-o).^  .^"(p-a)  da 

J0 


and  the  corresponding  element  in  H is  given  by 


(P 


.>(t-p+£)  ->(p-a)  da  f(s)ds,  0 < t,  < 
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and  is  locally  continuous  in  G £ C , for  any  /(• ) in  WQ(t). 

Hence  it  follows  tliat  x(t?co)  is  in  the  domain  of  C for  each  t 
and  w and  further  a simple  verification  establishes  (2.2)  since  the 
right  side  of  (2.2)  is  given  by 

V**  j, 

E [Cx(t,to)  yt(w')  ] [I  + L(t)  L(t)  ]-1 
and  for  any  f in  W^(t): 

E [Cx(t,w)  y^u)  ] f - C E [x(t,u>)  y^Gii)  ] f- 

Relation  (2.2)  enables  us  to  extend  the  arguments  in  [4]  to  show  that 


z(t,w)  = y(t,u)  - Cx(t,w)  0 < t < T 
is  again  white  noise.  Let  P^.(t)  denote 

E £ (x(t,ld)  - x( t )£jJ ) | ^x(t,w)  - X(t,(i))  j *J  . 

Ihen  P^(t)  = E [x(t,w)  xCtj-o)"]  - E Cx(t,oj)  x(t,w)*]  and  it  follows 

that  P^(t)  maps  into  the  domain  of  C.  The  covariance  operator  of 

y(*)  as  an  element  of  WQ(T)  has  the  form 

(I  + R) 

where  R is  Hilbert-Schmidt  and  hence  the  Krein  factorization  theorem 
(the  Kernels  being  strongly  continuous)  as  in  [4]  yields 
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(I+R)  = (1-Z‘j  (I-&) 


where  3!  is  volterra  and 


z(-,u)  = a-3)  y(*,u) 


Moreover 


(I- SO  1 = i + M 


where  M is  Hilbert-Schmidt  adso.  Hence  we  can  write 


x(* ,w)  = Tz(* ,w) 


where 


Tf  = g;  gCt)  = [ J(t,cr)  z(a,w)da 

J0 


and  following  [4]  we  must  have  that 


J(t,a)  = S(t-a)  (C  P^(a))" 


so  that 


Mt)x  = f S(o).^^*S(a)*xdcr  - f S(t-a)(C  P-(cx))*(C  P.(a))S*(t-a)dcr 

* Jo  J o f f 


and  in  turn  we  have  that,  for  x and  y in  the  doman  of  A* 
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[P^(t)x,y]  = CPf(t)x,A*y]  + [Pf(t)y,  A*x] 

+ [ Jfx,  &y~S  - [C  Pj.(t)  x,  C Pf(t)y];  (2.5) 


Pf(0)  = 0. 

Further  we  have: 


:,«)  = [* 
J n 


x(t,w)  = S(t-o)  (C  Pf(a))  (y(a,w)  - Cx(a,to))  da 


= -f 
J0 


S(t-a)  (C  P (a))“  C x(a,u)  da 


ft  * 

+ S(t-a)  (C  P,(a))  y(ff,u)  da 

j0  1 


(2.6) 


Ihis  is  an  'integral  equation'  that  x(t,w)  satisfies.  Moreover  (2.6) 
has  a unique  solution.  For  suppose  there  were  two  solutions  x^(t,co) , 

A 

Xj^tjto) . The  difference,  say  h(t),  (fixing  the  u)  ),  would  satisfy 

ft  * 

h(t)  = - S(t-c)  (C  Pf.(a))  C h(a)  da 
J0  1 


and  hence  we  can  deduce  that: 


C h(t)  = - f C S(t-o)  (C  P.(a))‘:(C  h(a) ) da 

Jo  1 

But  C h(*)  is  an  element  of  (0  ,T)  and  the  right-side  defines  a 
Hilbert-Schmidt  Vclterra  transformation  which  is  then  quasinilpotent.  Hence 
C h( • ) must  be  zero.  Hence 


-]  3- 


C x^(t,w)  = C x^(t,u>) 


Hence  z(t,w)  remains  the  same: 


z(t,w)  = y(t,w)  - C x^Ctju)  = y(t,uj)  - C x2(t,a>) 


But 


x(t 


,w)  = [ J(t, 


a)  z(a,w)ds 


proving  the  uniqueness  of  solution  of  (2.6).  We  could  also  have  deduced 
this  from  the  uniqueness  of  the  Krein  factorization.  We  can  also  rewrite 
(2.6)  in  the  differential  form  in  the  usual  sense  (see  [4]): 

t ~ 

x(t,co)  = A~(t,io)  + (C  Pf(t))  (y(t,u)  - C x(t,w)) 

x(0,w)  = 0 

yielding  thus  a generalization  of  the  Kalman  filter  equations.  Let  us  note 
in  passing  here  that 


A - (C  Pf(t))  C 


is  closed  on  the  domain  of  A and  the  resolvent  set  includes  the  open  right 
half  plane.  It  does  not  however  generate  a contraction  semigroup  for  t > 0. 

The  proof  of  uniqueness  of  solution  to  (2.5)  can  be  given  by  invoking  the 
dual  control  problem  analogous  to  the  case  where  C is  bounded,  as  in  [4]  but 
will  be  emitted  here  because  of  limitation  of  space.  Fran  this  it  will  also 
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follow  that  [P^-CtJxjx]  is  monotone  in  t. 


Let  C be  defined  on  H by: 


cnf  = g;  g(t)  = n 


1/n 


/(s)ds. 


Then  C is  bounded.  Hence  it  follows  that 


E (C  x( t 5cj) ) (C  x(t,m)*) 
n n 


= f (C  S( a).n  (C  S(a).>)*dcr, 
Jn  n n 


and  as  n goes  to  infinite,  the  left  side  converges  strongly  and  the  right 
side  yields 


C (C  R(t,f))*  ; R(t,t)  = E Cx(t,u)  x(t,u)*3. 


In  a similar  manner  we  can  show  that 


[ (C  x( t,to) ) (C  x(t,u))  = C (C  R(t,t))*  ; 

E Cx(t,io)  x(t,uj)*]  = R (t,t) 


E £ (c  (x  (t,cj)  - x(t,oj)  ) (c  xCt ,to)  - C x(t,to)]  *J 


= C (C  P^(t)) 


We  are  of  course  most  interested  in  the  case  T ■>  ®.  We  have 

seen  that  CP^(t)  x,x]  is  monotone.  Also 


[P,-(t)x,x]  [R  (t,t)x,x]  = | [S(a)->  .> ■"S(a)i’:x,x]da 


Let  us  assume  now  that 


j 1 1.>  *S(o)*x|  1 2 da  = [Rycjx]  < 00  (2 

(This  is  clearly  satisfied  in  our  example  (1.4)). 

Ihen  P^-(t)  also  converges  strongly,  to  P^,  say;  further  P 
maps  into  the  domain  of  C and  satisfies 

•CO 

Poo  = Rco  - • J s^)  (C  P my\c  PJ  S(a)*da 

and  hence  also  the  algebraic  equation: 

0 = [poox>A*y3  + (PqqI 5A"x]  + - [ C P^XjC  P^y]  (2. 

which  has  a unique  solution. 

3.  The  Control  Problem. 

Because  of  space  limitations,  we  shall  have  to  limit  the 
presentation  to  the  main  results,  emphasing  only  the  differences  arising 
due  to  the  unboundedness  of  C.  Thus,  defining  as  in  [4,  Chapter  6],  and 
confining  ourselves  to  controls  defined  by  (1.7); 

x(t,w)  - xu(t,w)  = x(t ,tj) 

C x(t,w)  + Gw(t)  = y(t,<j) 


where 


xu(t,cu)  = A x(t,u)  + B u(t,w) 
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r 


we  can  invoke  the  results  of  section  2 to  obtain  that 


— ...  z(t,w) -=-y(t,w)  - C x(t,w) 


where 


x(t,w)  = E [x(t,“)|  y(P,w),  0 < P < t] 

yields  v;hite  noise.  We  can  then  also  proceed  as  in  [4]  to  show  that  we  can 
also  express  any  u(t,w)  satisfying  (1.7),  also  as 


:,w)  = [ m(t,P)  z(P,u)dP 

j0 


where  the  operator  is  Hilbert-Schmidt . The  separation  theorem  follows 
easily  from  this,  and  we  can  show  that  -the  optimal  control  is  given  by 


■ - 1; 


(Q  C S(P-t)B)*  x(P,w)dP 


(3.1) 


wnere 


x (P,w)  = x (P,“>)  + xu(P,w) 


and  hence  as  in  section  2,  is  the  unique  solution  of 


x (P,u)  = A x(P,w)  + B un(P,w) 


+ (c  P^.(p))  (y(p,u)  - C x(p,w)) 


x (0,m)  = 0 


Further  we  can  follow  [4],  making  appropriate  modifications  of  the 
unboundedness  of  C,  to  deduce  from  (3.1)  that 

uQ(t,u>)  = - (P  (t)B)A  x(t,u)  (3.:: 

where  P (t)  is  the  solution  of 
c 

[P  (t)x,y]  = [P  (t)x,AyJ  + [P  (t)Ax,y] 

CO  c 

+ [QCx,  QCy] 

- [(Pc(t)B)*x,  (Pc(t)B)*y^ 

P (T)  = 0 0.2: 

c 

for  x,y  in  the  danain  of  A. 
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